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(57) A video encoder connprises an encoding sec- 
tion (28) for selectively performing on an input video sig- 
nal an intra-coding wo6e for intraframe coding, an Inter- 
coding mode for tnterframe coding, and a non-coded 
mode in which no coding is performed and the previous 
frame is used for display, a mode selector section (22) 
for adaptively selecting among the coding modes for 
each predetermined region in the input vkieo signal, 
and a refresh section (23) for detecting a motion region 
from within a frame and setting up an intraframe-coded 

Z4H 



region for refresh in a portion of a refresh range includ- 
ing the motion region. The refresh section determines 
which of a moticn region having motion and a still region 
with no motion each of intraframe-coded macrot)tock6 
belongs to. determines the refresh range in the next 
frame on the basis of the result of the d6terminatk>n, 
and instructs the intra mode to the mode select circuit 
when a macroblock to be coded belongs to the refresh 
range. 
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rooOl] The present invention relates to a video 
encoder and a video encoding method. More speS 
caHy^the presentinvention relates to a video eno^er 

are re^fS'p^T'"^ '^^-^es 
rZ^ '"^^'^"9 a" intraframe- 

m^^, ^^^^ °^ P*'^"^^ frames 

[M02] A Video codec (coder/decoder) has a prol^lem 

'T' T '""'^ '^'^ ^^^^^"^ - ^n^uni- 
caton channels and storage systems, picture data can- 
not be decoded correctly, causing deterioration "n 

aS^d?o"olT '°"-"*"^*« transmission channels 
and radio networks in particular, bit errors and cell loss 

:;rernr^"=*'^--------^^^^^ 

S On^Z^^'^ f""^ countermeasures against 
rXch 2 J"^ countermeasures is to 

tut fr^ m» ^"^^ ''^ '"^'^"9 '^^'^"^"y '"to the pic- 
L L ^L"^'^ °" "^'"^ *"*raframe coding 

s performed. Such a picture region can be d^3 
.ndependently of other frames and thus are not aISS 
by errors mixed in bit streams in the other f^rSS^ 

irrtraftame coding picture region cyclically into ea<S of 
P^cturejames a„«.s the error resilience to be 

is ISlnliT^'' ^ "^"^ "«^«^*ce 

coding mode (intra-mode) and interframe coding mode 

L?™r?^ ? °' P'^^'^ (macroblc^Tor 
example, an inlraslice-based refresh method has been 

Sa^S^^tj'Lf' ^ "^""^ '^"^^ ^ -"'^ 30 
macroWocks and honzontal 44 macroWocks a reaion 
^sting Of two macroblock lines (vert cS, S 
locks X honzontal 44 macroWocks) is encodedin STe 
";tra-mode. the Jntraframe-encoded region?2Sft^ bv 
[00051 In this intraslice scheme, when a search is 

ISf'ort fnf ^""^ '"^^'^ With a'^^* ; 

Snn ^1 ^ intraframe^ed 

[0006] For example, the search of motion vectors for 

macroWocks in the previous frame. macroblodSrajr 
responding locations to the intraframe-cod^^^n 
the current frame are not included in the mSo^J^^ 
search region. In the case of intraslice in vSiich^e 

o^'^ ^^''^^^ fr^"^ bo«o^ to top bect^me 

refreshed completely by, at the woret. the (2 cyclesTst 
frame from a frame in which errors have oS. r^l IS 
allow error propagation to converge. 



I0007J For example, assume that the cycle of 
intraslice is four frames and errors have occurred in the 

TmL "^^""^ ^y'^'e o' four 

. Sm! "^^^^^ «^i"g a macroblock in an input 

i^rr^rJ'^ 'T"'"'"- ^croblocks Which. Of 
macroblocks m the reference frame, are present in the 

to ^« tt^at macroblock 

tobe coded ,n the input frame are used for the motibn 
search. Thus, the range through whicr tSe 

!^o a'^<'^ propagate converges by the bottom- 
most region in the frame. """U'M 

mos] The number of coded bits in the intra-mode is 

/5 quantization parameters are used. 

'5 Since there are constraints on transmission rates the 
fransm.ss.on rate of bit streams is required to be limited 
to a certain value or less. When the intraslice-basS 
efresh scheme is used, therefore, it is required to 

^ ^Snrt"':r"°''^^^'*^'^>'*'^^~corr^ 
^ spending to the intraslice to be inserted by maWna 

SuTe^f S " "^f"^- ♦•''^ torms a serious 

cause of deteriorafion in picture quality. 

[0009] Recently, as disclosed in Japanese Unexam- 
•nedPatentPublicationNo9 - 2476^ywayofSm- 
Ple. several adaptive refresh schemes have^n 

SerSLtt"* f ^'^^'^ ^^^-'^ ^«9ions in a 
scene that have large motion to thereby reduce the 

T°"- ^""^ ^ncealment tech- 
SsJTl^ "^'"""^ '^'^ corresponding 
2!^^m f^m! P™;"°"« frame are embedded into the 
current frame. Thus, by refreshing adaptively onlv 
regions that are large in motion, it b4m^Sb,°e to 

ing the emor resilience. 

[0010] The conventional adaptive refresh schemes 
share a problem that the motion that infra cSS 
ZT,' ^ "°» considered. 

[0011] That is. the conventional adaptive refresh 
f".^ P""°P'« If^at, once a re^toTls 

rjs TncTr "^'"^ *^ '^^toct^^ 

Thus, once a region is coded in the intra mode it is 

"ew motion asso- 

aated with that region is detected 

S?„nl ^^"^ when a scene 

Jange occurs and the picture is still after the SSe 

Ji^lT^"^^ ^^'""^ °" <='^ange Sme 

d^out due to errors during transmission to the 
decoder, error-recovery is delayed, resulting a serious 
problem. This is illustrated in FIG 1 1 
mi3] inFIG. Il.itissupposedthatascenechange 
« occurs at frame N+1 and there is no motion for a^Se 
.n regions of macroblock MBi in succeeding frameT n 

rrr- *'ir*'=^""9 coding modes ^S a" 

the intra mode and the inter mode, is made adaptively 
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on a macroblock by macroblock basis according to the 
degree of temporal correlation in an input video signal. 
For example, when, like a scene change, a drastic 
change occurs in the picture, so that correlation 
between previous and current frames is lost completely. 5 
the intra mode is used. Thus, most of the maaoblocks 
in the scene change frame N+1 are coded in the intra 
mode. When the conventional type of adaptive refresh 
Is performed from top to bottom in a four-frame cyde 
from frame N+2, in frame N+2 the macroWock MBi is in ^ 
its turn to be refreshed as shown. However, since the 
macroblock MBi is coded in the intra mode in previous 
frame N+1 . the refresh of the macroblock MBi is inevita- 
bly skipped in frame N+2. In this case, the inter mode is 
chosen as the coding mode for the macroblock MBi in 75 
frame N+2. However, since there is no motion, the not- 
coded mode is actually used in which the corresponding 
macroblock in the previous frame is used for display 
without being coded. 

[001 4] After that. MBi's turn for refresh comes again in 20 
frame N+6. Unless motion is detected in the macroblock 
MBi by that time, the refresh of the macroblock is 
skipped in frame N+6 as well. 

[001 5] Thus, the macroblock MBi Is not refreshed for 
a long time. If, therefore, the macroblock MBi in the 25 
scene change frame N+1 drops out due to transmission 
errors, con-ect Information after scene change cannot 
be obtained for regions of the macroblock MBi. so that 
the picture in the previous frame N prior to the scene 
change is displayed as it is. 30 
[0016] As described above, the conventional 
intraslice-based refresh scheme, which increases con- 
siderably the amount of coding, requires to make quan- 
tization parameters large in order to attain coding at the 
same rate, degrading the picture quality. 35 
[001 7] Moreover, with the adaptive refresh schemes to 
refresh only motion regions, since no refresh is per- 
formed for a while after scene change, error-recovery is 
delayed in the event that errors are mixed in the scene 
change frame, degrading the quality of displayed pic- 40 
ture. 

[0018] Furthermore, when the nrrotion vector search 
range is restricted in order to ensure error-recovery by 
refresh, correct motion vectors cannot be found 
depending on the picture motion, degrading the picture 45 
quality. 

[0019] It is an object of the present invention to pro- 
vide a video coder and a video encoding method which 
permit the effects of transmission errors to be weakened 
and thereby the picture quality to be innproved without so 
increasing the anwunt of coding. 
[0020] According to an aspect of the present inven- 
tion, there is provided a video encoder comprising an 
encoding section for selectively performing an intra-cod- 
ing mode for intraframe coding of an input video signal, ss 
an inter-coding mode for interframe coding, and a non- 
coded nrxxle in which no coding is performed and the 
previous frame is used for display, a mode selector sec- 



tion for adaptively selecting among the coding modes 
for each predetermined region in the input video signal, 
and a refresh control section for detecting a motion 
region from within a frame and setting up an intra-coded 
region for refresh in a portion of a refresh range includ- 
ing the motion region, the refresh control section having 
a determination section for making a determination of. 
for each of intra-coded regions coded by the encoding 
section in the intra-coding niode, which of a motion 
region and a still region it belongs to and a refresh 
determination section responsive to the determination 
section for including intra-coded regions that belong to a 
motion region in the refresh range and excluding intra- 
coded regions that belong to a still region from the 
refresh range, and the encoding section intra-coding 
regions to be refreshed determined by the refresh deter- 
mination section. 

[0021] This video encoder supposes an adaptive 
refresh scheme that selects only a motion region within 
a frame as a refresh range and inserts an intra-coded 
region in part of the refresh range. That is, a still region 
is excluded from the refresh range. Thus, conrtpared 
with the normal intraslice. since the still region is not 
refreshed, the numt>er of macroblocks to be encoded in 
the intra mode is decreased and. at the same rate, the 
quantization parameters can be made small corre- 
spondingly, which helps improve the picture quality. 
[0022] Further, in determining the refresh range, 
unlike the conventional refresh, the motion of the intra- 
coded region on the frame is taken into consideration. 
[0023] That is, an examination is made of the degree 
of correlation between each of picture regions encoded 
in the intra modie and the corresporKfing picture region 
in the previous frame and a determination is made of 
which of a motion region having motion and a still region 
having no motion the picture region belongs to. Based 
on the result of this determination, the refresh range is 
determined. In this case, the intra-coded region that 
belongs to the motion region is included in the refresh 
range in the next frame and the intra-coded region that 
belongs to the still region is excluded from the refresh 
range in the next frame so that error-recovery can be 
made by refresh even if the intra-coded region that 
belortgs to the motion region drops out by errors during 
transmission to tiie decoder. Thus, by changing dynam- 
ically the refresh range on a picture region by picture 
region l>asis according to the presence or absence of 
motion of tiie intra-coded region in the previous frame, 
an intra-coded picture region. If it is a motion region, 
continues to be selected as the refresh range until it is 
later intra-coded again and considered to belong to a 
still region. Thus, in such situations as there is a scene 
change and the picture is still after the scene change, 
an intra-coded macroblock on the scene change frame 
is treated as a motion region and refreshed at a prede- 
termined time determined by the refresh cycle. Even if, 
therefore, an intra macroblock in the scene change 
frame drops out during transmission to the decoder. 
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error-recovery can be made relatively early based on 
refresh. 

[0024] The present invention further provides a video 
encoder which, when intra-coded regions that are con- 
sidered to belong to a motion region are included in one 
frame in a predetermined number or more, determines 
that the frame is a scene change frame and changes 
the location at which an intra-coded region for refresh is 
set up in the next frame so that a cyclic refresh opera- 
tion IS started at an end of the next frame. 
[0025] In general, the scene change frame contains a 
large amount of coding and hence has a high probability 
Of errors being mixed in. provided that the probability of 
occurrence of error remains unchanged. When errors 
are mixed in, since correlation between current and pre- 
vious frames is lost, a techniques, such as error con- 
cealment, cannot be used. By identifying a scene 
change and starting a refresh operation at an end of the 
frame, even if errors occur in the scene change frame it 
becomes possible to recover from the errors at the 
woret. in one refresh cycle of a predetemiined number 
of frames. 

[0026] In addition, the present invention provides a 
video encoder which, when a region in the next frame in 
the same location as a region in the previous frame 
where the intra-coded region for refresh has been set 
up IS encoded in the inter-coding mode, inhibits motion 
vector search, of motion vectors in the direction oppo- 
site to the direction of refresh, only for motion vectors 
from a region in the previous frame for which the deter- 
mination by the refresh control section is a motion 
region. 

[0027] To ensure errorrreoovery in (2 cycles - 1) times 
the conventional inti^slice scheme, as described previ- 
ously, wholly inhibits tiie search for motion vectors in the 
direction opposite to the direction of refresh. With ttie 
Still region there is littie possibility of errore being mixed 
in. Thus, by removing the still region from the motion 
vector search range, ttie motion vector search range is 
qCalJ^^ ^^"^ ^" improvement in pichire 

[0028] This summary of the invention does not neces- 
sanly describe all necessary features so tfiat the inven- 
hon may also be a sub-combination of these described 
features. 

[0029] The invention can be more fully understood 
from the following detailed description when taken in 
conjunction with the accompanying drawings, in which 
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operation in units of a fixed number of macroWocks: 
FIG. 6 is a diagram for use in explanation of the 
refresh operation of the video coder of FIG. i ; 
FIG. 7 is a diagram for use in explanation of the 
refresh operation of ttie video coder of FIG. 1 when 
there is a scene change; 

FIG. 8 is a diagram for use in explanation of restric- 
tions on the motion vector search range in ttie video 
coder of FIG. 1 . 

FIG. 9 is a schematic representation of a radio com- 
munication system to which ttie present invention is 
applied: ' 

FIG. 10 shows an example of an operational form of 
ttie radio communication system of FIG. 9; and 
FIG. 11 is a diagram for use in explanation of ttie 
operation of a conventional adaptive refresh 
scheme. 



FIG. 1 is a block diagram of a video coder according 
to a first embodiment of ttie present invention: 
FIG. 2 is a flowchart for ttie coding process in the 
video coder of FIG. i; 

FIG. 3 is a schematic representation of a refresh 

operation using horizontal macroWock lines; 

FIG. 4 is a schematic representation of a refresh 

operation using vertical macroblock lines; 

FIG. 5 is a schematic representation of a refresh 



[0030] In FIG. 1, an input video signal is divided into 
20 macroWocks in a blocking circuit 11. The input video 
signal having divided into macroWocks is entered into a 
subtracter 12 where ttie input video signal and a predic- 
tion video signal to be described later are subtracted 
and their difference yields a prediction residual error 
25 signal. Eittier of ttie prediction residual signal and ttie 
input video signal from ttie blocking circuit 1 1 is selected 
ty a mode selection switch 13 and ttien subjected to 
discrete cosine ti^nsform in a DCT circuit 14 DCT coef- 
ficient data obtained in ttie DCT circuit 14 are quantized 
30 in a quantization circuit 15. The output of ttie quantiza- 
tion arcuit 15 is divided into two: one is variable-lengtti 
coded in a variable lengtti coder 25. 
[0031] The ottier of the divided outputs of the quanti- 
zation circuit 15 is dequantized and inverse DCT coded 
n^H^ ^ dequantization circuit 16 and inverse 
DCT (IDCT) arcuit 1 7 and tfien added in an adder 18 to 
a prediction picture signal from a switch 21 , thereby pro- 
duang a local decoded signal. The localdecoded signal 
IS stored into a frame memory 19 and ttien fed into a 
40 motion compensation circuit 20. The motion compensa- 
tion arcuit perfomis tfie motion compensation process 
such as motion detection and motion vector search oi^ 
ttie basis of a)rrelation between ttie input video signal 
and tfie video signal in the previous frame stored in ttie 
« frame memory 19, producing ttie prediction video signal 
and sending necessary information to a mode selector 
22 and a refresh determination circuit 23. 
[0032] An encoding conti-ol circuit 24. used for rate 
.n f!"*;°I ~^°'^«^'"9°" the basis Of ojdedinforma- 
50 fon from an encoder section 28 and ttie buffer amount 
of an output buffer 17 and sends necessary information 
to ttie vanaWe lengtti encoder circuit 25. In ttie variable 
Srf^^ u*'"''^'* «>efficient information is 

« . information and 

55 mode information MODE. Image data produced by ttie 
coding is sent to a multiplexer circuit 26 where it is a>m- 

hZ "^^^ «her encoded 

data and ttie resulting multiplexed signal is ttien trans- 
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<^ mitted as cx)ded data (bit stream) over a transmission 

channel with the transmission rate smoothed in the out- 
put buffer 27. 

[0033] The mode selector circuit 22 makes an adap- 
tive selection between macroblocks for interframe cod- 
ing arKJ macroblocks for Intraframe coding based on 
prediction information P from the motion compensation 
circuit 20 which indicates the degree of intermacroblock 
correlation. 

[0034] For intraframe coding, the switch 1 3 is switched 
to terminal A by mode selection switch information M 
and the switch 21 is switched to terminal A by switch 
information S. For interframe coding, on the other hand, 
the switch 1 3 is switched to terminal B and the switch 21 
is switched to terminal B. 

[0035] Coding modes include sin intra-mode (INTRA) 
for intraframe coding, an inter-mode (INTER) for inter- 
frame coding, and a non-coding mode (NOT_CODED) 
in which con^esponding macroblock information in the 
previous frame is used as it is for display. A selection is 
made adaptively among these coding modes on a mac- 
roblock-by-macroblock basis. Normally, the INTRA 
mode is selected in the case where there is no inter- 
frame correlation. Where interframe correlation can be 
employed for prediction, the INTER mode is selected. 
Where there is no change in picture and hence coding 
is unnecessary, the NOT_CODED mode is selected. 
[0036] In addition, when the refresh determination cir- 
cuit 23 issues an instruction to perform refresh, the 
mode selection circuit 23 selects the INTRA mode. 
[0037] The mode select information MODE indicating 
a coding mode selected by the mode select circuit 22 is 
sent to the variable length encoding drcutt 25 arKi the 
refresh determination circuit 23. The refresh determina- 
tion circuit makes a determination as to whether each 
macroblock is to be included In a refresh region or not 
on the basis of the selected coding mode indicated by 
the mode select mode MODE and the presence or 
absence of motion in intraframe-coded macroblocks. 
Based on the result of this determination, whether 
refresh is to be performed or not is determined. 
[0038] That is, the present emtKXIIment supposes the 
adaptive refresh scheme that selects only nrK>tion 
regions in each frame as a refresh range. However, in 
determining the refresh range, unlike the conventional 
adaptive refresh scheme, the nK>tion of picture across 
frames at each intra macrot>lock coded in the INTRA 
mode is considered. In this case, for an intra macrob- 
lock for which the determination is that it belongs to a 
motion region, its region is included in the refresh range 
in the next frame so that the refresh will provide recov- 
ery from errors even if the macroblock itself drops out by 
errors during transmission to the decoder. An intra mac- 
roblock for which the determination is that it belongs a 
still region is excluded from the refresh range in the next 
frame. In this manner, the refresh range in the next 
frame is dynamically changed on a macroblock-by-mac- 
roblock basis according to the presence or absence of 



motion of picture at each intra macroblock in the previ- 
ous frame. 

[0039] FIG. 2 is a flowchart for the processing opera- 
tion of the refresh determination circuit 23, which illus- 
5 trates the operation of performing cyclic refreshing by 
shifting the INTRA coding region from top to bottom in 
units of horizontal one macroblock line from frame to 
frame. 

[0040] Here, i and j represent addresses in the vertical 

10 and horizontal directions, resjDectively. in the frame of a 
macroblock to be coded. V_NMB and H_NMB repre- 
sent the numbers of macroblocks in the vertical and hor- 
izontal directions, respectively, in the frame. N is the 
count in a counter indicating a macroblock line at a 

75 refresh location. 

[0041] A two-dimensional array D[i][j] contains ele- 
ments for managing refresh control information indicat- 
ing for each macroblock whether it belongs to a motk}n 
region needing refresh or not on the basis of the hiistory 

20 of motion region/still region determinations for each 
macroUock. The refresh control information for a mac- 
rok^lock on which rewriting has been performed, namely, 
a macroblock for which the determination is motion 
region because of the presence of a picture change, is 

25 set to FALSE. The refresh control information for a mac- 
roblock for which the determination is that there is no 
change in picture is set to TRUE. 
[0042] The FALSE indicates that the corresponding 
macrot^lock currently belong to a motion region. Unless 

30 the FALSE is rewritten to the TRUE, that macroblock is 
refreshed in a specific frame determined by the refresh 
cycle. 

[0043] The counter SC. provided in the refresh deter- 
mination circuit 23. counts the number of INTRA mac- 
35 roblocks for which the determination is motion region for 
the purpose of determination of scene change. 
[0044] First, before starting coding, the counter value 
N to find the refresh location is initialized to 0 (step 
S101). 

40 [0045] Next, the counter SC for scene change deter- 
mination is initialized to 0 before an input 1 freime is 
coded (step SI 02). 

[0046] In loop 1 and loop 2. the refresh determination 
and the coding process are performed for each nnacrob- 

45 lock in the frame. In loop 2 composed of steps SI 04 
though S1 14, processing is performed on the same 
macroblock tine in the horizontal direction. By nesting 
loop 1 inside loop 1 composed of steps SI 03 arid S104. 
the macroblock line which is an object of processing is 

50 Shifted from top to bottom, thereby processing one 
frame in its entirety. 

[0047] In step SI 05. an examination is made as to 
whether or not two conditions are met: (1 ) the remainder 
of division of the refresh counter value N by the number 
55 of macroblocks in the vertical direction, V_NMB, is 
equal to address i of a macroblock that is currently an 
object of coding; arxj (2) the value of array D [i]0] is 
FALSE. If the macroblock that is going to be coded 



5 



EP 0 933 948 A2 



belongs to the macroblock line to be refreshed then N 
moduto V^NMB is equal to i. the address o, the n4<^ 
^ tha ,s going to be coded. In addition, if the array 
DRffl value ,s FALSE, then i, is decided that the macrob 

S t"^!^ T'H^ '° ''^ '° « "lotion region 

that needs to be refreshed. 

[0048] If, merefore. N modulo V_NMB equals the 
a^d r ' °' that is going ,o be coded 

srty of refr^h. then the refresh is determined and the 

SSnf^^'"*""^""^ rewritten to the 

INTRA mode (step Si 06). 

"^'"3 P'°<=®^ (st^ SI 07). a coding 
mode determination is made (step S108) and then the 

array DHD] is updated by coding mode type ""^"'^^ 
[00501 If the MODE is INTRA, then in order to deter- 
mine whether the macroblock is still, the sum (SAD) of 

Srif!J^T °' ^'^^^^ '^''^"^'s °* the macrob- 
™ J from the motion 

Sfr^K 'If ^° °) compared with a 

Tdl^ / J'^ ^" examination is 

STtra 1 w« '^^'^^ '''^ macroblock coded in the 
^ '^'■^ "macroblock of the same 

address in the previous frame. If SAD is larger than TH. 
«ie deterniination is that the picture is in motion. In this 

FALSE indicating the necessity of refresh (step 8110) 
and the count in the counter SC for scene change deter- 
mination IS incremented by one (step Sill) On the 
t^t'ih^m • "!iS;^.'-'-9-t'^ai TK itisdetef^inS 
that the macroblock is still. In this case, the correspond- 
ing array DHOJ value is set to TRUE, indica^^, 
there IS no need of refresh (step S1 12) 
[0051] If the MODE is INTER, it is decided that there 
.s motiorj in the macroblock and then the coirCiSfng 
array D[,TQJ value is set to FALSE indicating 
sity of refresh (step S1 13). yue neces 

[0052] If the MODE is NOT CODED the anav Dmm 
.sr^jup^atedand the opeiation goes to*^^^^^^ 

M rJ^ rewritteruo 
TRUE only when a macroblock in a still reqion is 

I'-^^'^^J^"* « that aTc^^ 

lock coded in the INTRA mode belongs to a ^^on 
region or when a macroblock is coded in the ^ 
mode the array D[i]DJ value is kept at FALSE and the 
macroblock continues to be selected as a refresh range 
unti the macroblock at the same location in the J^- 
SS? !!r^ !! "«<=^°W°ck is coded again In the 
^'^ «*«termined that it belongs a still 

[OOM] After the termination of loop 1 and loop 2 that 

value SC for scene change detemiination is compared 
w* a threshold E (step S116). If SC > E. it i^SS 
that the frame for which coding has completed is not a 

Ss^tfi"^^ ? ^ ^^"^ this 

by V_NMB and the remainder is set as a new value for 
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N (st^ S1 17). If. on the other hand. SC is not greater 

^ml. J '^''"^ '^^^ ^'hich coding has 

completed is a scene change frame. In this case N is 
reset to 0 (step Si 18). thereby allowing the refresh 
5 operation to be restarted from the top of th'e next frame 
W055] In the coding process (step S107). the motion 
oomf^nsaton circuit 20 inhibits motion compensation 
from a motion region among un-refreshed regLs in the 

[0056] In the above description, the refresh operation 
IS performed based on horizontal macroblock Hnes as 
shown in FIG. 3. Alternatively, as shown in FIG 4 t^e 
refresh operation may be perfbmied based on vei^ical 
macroblock lines. In such a case, it is required onl^^S 
^1 °' ^ ^-'^'^^ ^«P« S105 and S1 17 be 

TrSf °!i:«'=^°Wocks in the horizontal direction 
[M57] That is. with vertical refreshing, in step S105 
detemiinations of (J:N mod H_NMB) and (Dfiini-FALSB 
:;fPfrf°^"'«* 'n other words, a cieckL 
whether or not the two conditions are both met: (1) the 
remainder of division of flie refresh counter value N by 
fte number of rnacroblocks in the horizontal direction 
25 nblf;K*'"!^*°**^'^j°'«'"acroblockthatis 
Zm , ^l"^^' ^2) the corresponding airay 
DHO] v«lue has been set to FALSE. When the macrci 
Ode that IS going to be coded belongs to a macroblock 

H MMR T^*"^- '^"^"^^^ °t division of N by 
the address i of that macrobk)ck 

!l""*ri^^°ra[n value has been set t^ FAL^t 

""^t'l^is^r'^--^^^^^'^-^^'-^-^ 

[0058] Thus, when, in step S105. N mod H NMB Is 

MnnirT, ^ ' information 

or2«r *° "^f^ "^e. so that a refresh 

operation is performed. 

I0059J In the above example, the refresh is performed 
•;°"«»^al line or vertkal line basis. AlternatiSt, 
the refr^h may be peribrmed in units of a preirS 

/ T^^- ^^«™"ations are made as to (I x 

S«H ^^'^"y- the above refresh mode is 
the refresh can be performed in units of a predeter- 

^7«dn"mber(RxN)ofmacroblocksasshov^rFiG 
^'®P S105. ((i X V NMB + i)/RN • N 
n«d(V NMB X H_NMB)/RN) and (Dfoin : FASare 
fnorrS2 the value'oSlalned^^Ji^;' 

2^^^^ ^' ""-""^^^^ °* macroblock to be 
« i^S^l !L'^!*"''"'■"^"""*«^^f'''N)indi- 
« rates an order in which the region is to be refreshed It 

vlue^lftot"^" " 

v^ue equals to the remainder obtained by the value of 
the refresh counter N by the number ((V_NMB x 
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H_NMB)/RN) of regions to be refreshed and the other 
that the value of the array D[l]0] is "FALSE", are estab- 
lished. If the macroblock to be currently encoded 
belongs to the macroblock to be refreshed, the value 
obtained by dividing the address (i x V_NMB+j) of the 5 
macroblock to be encoded by the predetermined 
number (RN) equals to the remainder obtained by divid- 
ing the value of the counter N by the number ((V_NMB 
X H_NMB)/RN) of regions to be refreshed within the 
frame. Also, if the value of the an*ay D[l][fl is "FALSE", it w 
is determined that the corresponding macroblock to be 
encoded belongs to the motion region to be refreshed. 
[0061] Therefore, in step SI 06. if the value obtained 
by dividing the address (i x V_NMB+i) of the macroblock 
to be encoded by the predetermined number RN equals is 
to the remainder obtained by dividing the value of the 
counter N by the number ((V_NMB x H_NMB)/RN) of 
regions to be refreshed within the frame, and the value 
of the array D[I]D1 is "FALSE" indicating the necessity of 
the refresh, it is determined that the refresh must be 20 
performed, and then the encoding mode information 
MODE is rewritten to the INTRA nrvxie. 
[0062] As described above, in this embodiment, 
whether to refresh a macroblock is determined based 
on whether it has been rewritten as a result of a change 25 
in picture after it was last refreshed. At this point, the 
important thing is to determine whether, if an INTRA 
coding macroblock. including macroblocks that have 
been refreshed previously, is coded in the INTER mode, 
it becomes NOT_CODED. If it can be determined so. 30 
the INTRA coding macr(±riock belongs to a still region. 
Even if it drops out. the effects can be suppressed by 
using the corresponding macroblock in the immediately 
preceding frame as it is. If not. the macroblock is 
present in a motion region. Thus, if it drops out. then the 3S 
effects become serious. If the macroblock is present in 
a still region and has not been rewritten, it needs to be 
refreshed. 

[0063] FIG. 6 shows a refresh operation for horizontal 
macroblock lines in frame sequence. In this example, it 40 
is assumed that the cycle period of refresh is four 
frames and the refresh operation is performed from top 
to bottom of the frame. 

[0064] In FIG. 6. subsequent to the INTRA macrob- 
lock MBi in a still region refreshed in frame N, 45 
NOT_CODED macroblocks. which belong to a still 
region, continue through four consecutive frames corre- 
sponding to the cycle of refresh. In this case, in frame 
N+1 . the corresp<»iding array D[i][0 value is maintained 
at TRUE indicating that there is no need of refresh; thus, so 
the refresh of the macroblock MBI in frame N+1 is 
skipped. Likewise, since the INTRA macroblock Mni+3 
in a still region refreshed in frame N-k3 is followed by 
NOT CODED macroblocks through frame N+6. the 
macroblock MBi+3 in frame N+7 is skipped. However, ss 
even if the macroblock MBi+3 is skipped in frame N+7, 
since it is coded in the INTER mode, it will be refreshed 
in the next refresh cycle. 



[0065] Therefore, this embodiment ensures recovery 
from errors in. at the worst, a (2 cycles - 1) number of 
frames. 

[0066] FIG. 7 shows a refresh operation when there is 
a scene change. 

[0067] When a scene change occurs in frame N+1 . 
substantially all macroblocks in frame N+1 are coded in 
the INTRA mode. In this case, since each INTRA mac- 
roblock is present in a motion region, the corresponding 
array D[i]0] value is set to FALSE, indicating the neces- 
sity of refresh. Thus, even if the intra macroblocks in 
frame N+1 drop out due to errors during transmission to 
the decoder, error-recovery can be made relatively early 
after that. 

[0068] In the present emkxxdiment. as described pre- 
viously, when It is determined that a frame is a scene 
change frame after it has been coded, the refresh loca- 
tion is reset to the initial value in the next frame so that 
the cyclic refresh is started at the top of the next frame. 
In the case of refresh from top to bottom, therefore, the 
refresh is started at the topnrxjst macroblock line in 
frame N+2 following the sc^e change frame N+1 . 
[0069] In general, the frame in which a scene change 
occurs involves a large amount of coding and hence has 
a high probability of error being mixed in under the 
same probability of occurrence of error. In case where 
errors are mixed in. an effective technique, such as con- 
cealment, cannot be used due to the absence of corre- 
lation with the previous frame. When no scene change 
is particularly considered, the refresh-based recovery 
will take, at the worst, a (2 cycles - 1) number of frames. 
However, by starting the refresh from the top of the 
frame immediately after scene change as in this embod- 
iment, recovery can be made, at the worst, in one cycle 
as shown in FIG. 7 even if the scene change frame con- 
tains errors. 

[0070] FIG. 8 shows a relationship between INTRA 
coding regions inserted by refresh and still regions. 
[0071] As described previously, in searching for the 
motion vector associated with a macroblock in a region 

1 above an intra coding region in the current frame N+2. 
the conventional system restricts the search range from 
the previous frame N+1 serving as the reference frame 
and inhibits nrotion compensation from the entire region 

2 which has not been refreshed yet. 

[0072] However, for the still region, there is little need 
to inhibit motion compensation because there is little 
possibility of errors being mixed in. In the present 
embodiment, therefore, in order to ligfrten restrictions 
on the motion vector search range and thereby improve 
the picture quality, the motion vector search is permitted 
with respect to still regions (J1 in frame N. j2 in frame 
N+1 . and J3 in frame N+2) in the region 2. This is imple- 
mented by allowing the refresh determination circuit 23 
to examine motion regions and still regions in the region 
2 in the reference frame using the array D[i][jl and 
instruct the motion compensation circuit 20 to inhibit the 
search of only the motion regions. The direction of 



7 



13 



EP 0 933 948 A2 



14 



refresh may be not only from top to bottom but also from 
bottom to top or from left to rigtit. In ttiis case it is 
required to intiibit ttie search for. of motion vectors in the 
direction opposite to the direction of refresh, only motion 
vectors from regions considered as motion regions. 5 
[0073] The first embodiment has been described as 
updating the an-ay Dm] to the value TRUE indicating 
that there is no need of refresh as soon as a still IMTRA 
macroblock appears. The conditions may be made 
more stnngent such that, if a still INTRA maaoblock 10 
appears n times (n > 1), it may be considered to belong 
to a still region. In this case, even if an INTRA macrob- 
lock at a location drops out in n consecutive frames 
error-recovery can be made by refresh. 
[0074] In addition, although the embodiment has been is 
desaibed in terms of coding on a frame basis, the pre- 
set invention is likewise to ^licable to coding on a field 
t^asis. 

[0075] Next, a second embodiment of the present 
invention will be described below. 20 
[0076] FIG. 9 shows a configuration of a radio commu- 
nication system using the video coder of the present 
invention. The radio communication system includes a 
video transmission system 120 and a video reconstruc- 
tion system 130. Picture information is transfeired from 25 
the video transmission system to the video reconstruc- 
tion system over a network 140 provided with a base 
station 141. 

[0077] The video transmission system 120 comprises 
a video signal input section 121 . an information source 30 
encoder section 122 having a robust processor section 
123, a transmission line encoder section 123, a trans- 
mission line encoder section 124. and a radio section 
125. In the infbnuation source encoder section motion 
compensation, discrete cosine transform (DCT). quanti- 3S 
zation and so on are performed. In the transmission line 
encoder section, error detectton and correction 
processing is performed on coded data The infbrmatron 
source encoder section has the video coder of FIG. 1 
and the robust processor implements the above- 4o 
descnbed refresh control functions. 
[0078] The video reconstruction system 130 com- 
pnses a radio section 131. an information source 
decoder section 133 having a robust processor 134 
and a video signal output sectran 135. The information 45 
source decoder section is adapted to decode bit 
sfreams obtained by the video coder of FIG 1 The 
robust processor 134 has error concealment functions 
[0079] FIG. 1 0 shows a form of operation of the radio 
communication system according to the second embod- so 
mem. As shown, video information is transmitted 
among terminals 50, such as laptop computere 151 and 
desWop computers 1 52, over a communication network 
140 having base stations 141, 142, and 143. 
10080] Picture information captured by a camera 151 a 55 
as a picture information input section installed in the lap- 
top computer 1 51 is coded by the information coder built 
into ttie computer. The coded data outputted from the 



information source encoder section and other speech 
information and data are multiplexed togettier and then 
transmitted by radio from the radio section and an 
antenna 151b built into ttie computer. The transmitted 
information is sent to the network 140 tfirough a near 
one of the base stations 141 to 143 to the network 140 
and then received by the antenna 152a of the destina- 
tion computer 152 through a near one of the base sta- 
tions. The received information is demultiplexed into ttie 
picture coded information, speech information and data 
The picture coded information is coded by the informa- 
tion source depoder section built into the computer and 
then displayed on the display of the computer. 
JMI8I] Picture information captured by the camera 
1Kb IS coded in the same manner as above using ttie 
infomiation source coder section. The coded informa- 
tion and other speech information and data are multi- 
plexed together and then transmitted by radio from tiie 
antenna 1S2a. The transmitted radio signal is received 
by the antenna 151a through the network 140 The 
received information is demultiplexed into the picture 
coded information and other speech information and 
data. The picture coded information is decoded by tiie 
information source decoder section in the computer and 
then displayed on its display 

[0082] As described above, accoiding to tiie present 
invention, by implementing ttie adaptive refresh opera- 
tion of determining ttie refresh range on tfie basis of a 
determination of whether an irtra-coded picture region 
IS eittier a motion region or a still region, ttie effects of 
bansmission errors can be lessened wittiout increasing 
ttie amount of coding, ttius allowing the pirture quality to 
be increased. In addition, by resetting the refresh toca- 
tion to ttie initial location in the frame immediately fol- 
lowing a scene change frame, early error-recovery can 
be made in the even of ttie dropout of ttie scene change 
frame due to transmission en-ors. 

Claims 



1. 



A video encoder characterized by comprising: 

an encoding section (28) for selectively per- 
fomiing an inti-a-coding mode for intraframe 
coding of an input video signal, an inter-coding 
mode for interframe coding, and a non-coded 
mode in which no coding is performed and ttie 
previous frame is used for display; 
a mode selector section (22) for adaptively 
selecting one coding mode among ttie coding 
modes for each predetermined region in ttie 
input video signal; and 

a refresh control section (23) for detecting a 
motion region from witiiin a frame and setting 
an intra-coded region for refresh in a portion 
of a refresh range including tiie motion region 
the refresh conti-ol section having a determina- 
tion section for making a determination of for 
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each of intra-coded regions coded by the 
encoding section in the intra-coding mode, 
which of a motion region and a still region it 
belongs to and a refresh determination section 
responsive to the determination section for § 
, including intra-coded regions that belong to a 
motion region in the refresh range and exclud- 
ing intra-coded regions that belong to a still 
region from the refresh range, and 
the encoding section intra-coding regions to be io 
refreshed determined by the refresh determi- 
nation section. 



2. The video encoder according to claim 1 , character- 
ized in that the refresh control section (23). when 75 
the nunr^er of intra-coded regions that are consid- 
ered to belong to a motion region are included in 
one frame in a predetermined number or more, 
determines that the frame is a scene change frame 
and changes the location at which an intra-coded 20 
region for refresh is set up in the next frame so that 

a cyclic refresh operation is started at an end of the 
next frame. 

3. The video encoder according to claim 1 . character- 2s 
ized in that the refresh control section (23) shifts the 
location at which the intra-coded region for refresh 

is set up cyclically for each frame. 

4. The video encoder according to claim 3, character- 30 
ized in that the refresh control section (23) shifts the 
intra-coded region in the vertical direction of frame 

on a maaobtock line basis for each frame. 

5. The video encoder according to claim 3, character- 3S 
ized in that the refresh control section (23) shifts the 
intra-coded region in the horizontal direction of 
frame on a macroksiock line basis for each frame. 

6. The video encoder according to claim 3, character- 40 
ized in that the refresh control section (23) shifts tiie 
intra-coded region in the horizontal direction of 
frame in units of a predetermined number of mac- 
roblocks for each frame. 

45 

7. The video encoder according to claim 3, character- 
ized in that the refresh control section (23). when a 
predetermined number of intra-coded regions that 
belong to a motion region or more are included in 
one frame, determines that the frame is a scene so 
change frame and changes tiie location at which 

the intra-coded region for refresh is set up in the 
next frame so that a cyclic refresh operation is 
started at an end of the next frame. 

55 

8. The video encoder according to claim 3, character- 
ized in that said refresh control section includes an 
inhibit section for, when a region in the next frame in 



the same location as a region in the previous frame 
where the intra-coded region for refresh has been 
set up is encoded in the inter-coding mode, inhibit- 
ing motion vector search, of motion vectors in ttie 
direction opposite to the direction of refresh, only 
for motion vectors from a region in the previous 
frame for which the determination by the refresh 
control section is a motion region. 

9. The video coder according to claim 1 , characterized 
in that the refresh control section (23) determines 
that each of inter-coded regions encoded in the 
inter-cbding mode by the encoding section is a 
motion region arxl determines the refresh range so 
that each inter-coded region is included in the 
refresh range. 

10. The video encoder according to claim 1 , character- 
ized in that said encoder section includes a motion 
compensation circuit for detecting motion on a mac- 
roblock by macroblock basis from correlation 
between current and previous frames to generate 
prediction information, arxj wherein the mode 
selection circuit (22) makes an adaptive selection 
between a macroblock to be encoded in the inter- 
coding mode and a macroblock to be encoded in 
the intra^xxiing mode on the basis of the prediction 
information irvjicating the degree of the correlation. 

1 1 . A video encoder characterized by comprising: 

an encoding section (28) for selectively per- 
forming an intra-coding mode for intraframe 
coding of an input video signal, an inter-coding 
mode for interframe coding, and a non-coding 
mode in which no coding is performed and the 
previous frame is used for display; 
a mode selector section (22) for adaptively 
selecting among the coding modes for each 
predetermined region in the input video signal; 
and 

a refresh control section (23) for detecting a 
motion region from within a frame and setting 
up an intra-coded region for refresh in a portion 
of a refresh range including the motion region, 
wherein 

the refresh control section detects the motion 
region on the basis of the selected coding 
mode for each picture region and the determi- 
nation of whether an intra-coded region 
belongs to a motion region, and 
the encoding section intra-codes regions to be 
refreshed determined by the refresh determi- 
nation section. 

12. A video encoding method characterized by com- 
prising the steps of: 
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13. 



selectively encoding an input video signal in an 
intraframe coding mode, an interframe coding 
mode, and a non-coding mode in which no cod- 
ing IS performed and the previous frame is 
used for display: 

adaptively selecting among these coding 
modes for each picture region in the input video 
signal: 

detecting a motion regron from within a frame 
and setting up an intraframe coded region for io 
refresh in a portion of a refresh range consist- 
ing of the motion region: 
determining, for each intraframe coded region 
which of a motion region and a still region it 
belongs to: and 

determining the refresh range in the next frame 
so that infraframe coded regions that belong to 
the motion region are included in the refresh 
range and intraframe coded regions that 
belong to the still region are excluded from the 20 
refresh range. 

The video encoding method according to daim 12 
characterized by further comprising the step of' 

when a predetermined number of infraframe-coded zs 
regions that belong to a motion region or more are 
included in one frame, determining that the frame is 
a scene change frame and changing the location at 
which the intraframe-coded region for refresh is set 
up in the next frame so that a cyclic refresh opera- 30 
tion IS started at an end of the next frame. 

14. The video encoding method according to daim 12 
characterized by ftjrther comprising the step of 
shifting the location of the intraframe-coded region 35 
for refresh cydically for each frame. 

15. The video encoding method according to daim 14 
characterized in that the intraframe^ed region is 
shrfted in the vertical direction of frame on a mac- 4o 
roblock line basis for each frame. 

16. The video encoding method according to daim 14 
characterized by further comprising the step of' 

when a predetennined number of intraframe^xxled « 
regions that belong to a motion regron or more are 
induded in one frame, determining that the frame is 
a scene change frame and dianging the location at 
which the intra-coded regron for refresh is set up in 
the nact frame so that a cyclic refresh operation is so 
started at an end of the next frame. 

1 7. The video encoding method accoitiing to daim 14 
characterized by comprising, when interframe cod- 
ing a region in the next frame in the same location ss 
as a region in the previous frame where the intra- 
frame-coded region for refresh has been set up 
inhibiting motion vector search, of motion vedors in 



the direction opposite to the direction of refresh 
only for motion vectors from a region in the previous 
frame for which the determination is a motion 
region. 

18. The video encoding method according to claim 12 
diaracterized in that the refresh range determina- 
tion step includes determining that each of inter- 
frame-coded regions is a motion region and 
determining the refresh range so that eadi of the 
interframe-coded regions is induded in the refresh 
range. 

19. A video encoding method characterized by com- 
pnsing the steps of: 

selectively encoding an input video signal in an 
intraframe coding mode, an interframe coding 
mode, and a non-coding mode in which no cod- 
ing IS performed and the previous frame is 
used for display: 

adaptively selecting among these coding 
modes for each picture region in the input video 
signal: 

detecting a motion region from within a frame 
and setting up an intraframe coded region for 
refresh in a portion of the motion region- and 
detwting a motion region in the next frame on 
the basis of the selected coding mode for each 
picture region and the determination of 
whether an intraframe-coded region belongs to 
a motion region. 
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BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of the original 
documents submitted by the appHcant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 
^□^ApED TEXT OR DRAWING 
U^LURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



